Table of Contents
In this notebook we explore data from the Medalla testnet. We are looking at the 20161 first slots.
Data sources
Lighthouse block export
We use a fork of Lakshman Sankar’s Lighthouse block exporter to export attestations and blocks from the finalised chain until slot 20160.
We present the main datasets below:
logical_ats
Each row in this dataset corresponds to an aggregate attestation included in a block. We cast its attesting_indices (the bits recording which validators are aggregated in this aggregate attestation) to a wide format, with one column per validator. This is shown below (columns past column “3” are truncated).
exploded_ats
We cast the dataset above into a long format, such that each row corresponds to an individual attestation included in a block. Note that when this individual attestation is included multiple times over multiple aggregates, it appears multiple times in the dataset.
all_dps
Validators are allowed to deposit more ETH into their eth2 accounts. Later on, we compute the reward obtained by a validator by comparing its initial balance with its current balance. We must then deduct deposits that were made since genesis.
Weald dump
Jim McDonald, from Attestant, kindly provided a treasure trove of data on the #medalla-data-challenge channel of the EthStaker Discord server. The two previous datasets could have legitimately been mined from Jim’s data, but we like to get our hands dirty.
all_cms
Not too dirty though: obtaining the past record of committees (which validators are supposed to attend when) is much more computationally intensive, since it requires access to past states. This dataset is a long-format record of which validators appear in which committee and when.
val_balances
The final dataset gives us validator state balances at the end of each epoch. Note that the state balance, the true ETH amount a validator owns, is different from effective balances, which measure the principal on which validators receive an interest.
Validator performance
Validators are rewarded for their performance, and penalised for failing to complete their tasks. We start with a crude measure of performance: the number of included attestations. It is a crude measure since (a) we do not discount the timeliness of the validator, measured by the inclusion delay and (b) we do not check that the attestation’s attributes are correct (with the exception of the source attribute, since an incorrect source cannot possibly be included on-chain).
Uptime-rewards curve I: Included attestations
We compare the percentage of included attestations with the (possibly negative) reward obtained by the validator.
Uptime-rewards curve II: Inclusion delay
We turn our attention to the inclusion delay. Validators are rewarded for attesting timely, with higher rewards the earlier they are included in a block. We explode aggregates contained in the blocks to trace the earliest included attestation of each validator in an epoch.
Note that the y axis is given on a logarithmic scale. A high number of attestations have a low inclusion delay, which is good! Since attestations cannot be included more than 32 slots from their attesting slot, the distribution above is naturally capped at 32.
How is the inclusion delay correlated with the rewards? We look at validators who have the highest number of included attestations to find out.
Correctness of attestations
Aggregate attestations
eth2 is built to scale to tens of thousands of validators. This introduces overhead from message passing (and inclusion) when these validators are asked to vote on the canonical chain. To alleviate the beacon chain, votes (a.k.a. attestations) can be aggregated.
In particular, an attestation contains four attributes:
- The slot it is attesting for.
- Its vote for the head of the beacon chain, given by the fork choice rule.
- Its vote for the source, i.e., the last justified checkpoint in its view.
- Its vote for the target, i.e., the checkpoint to be justified in its view.
Since we expect validators to broadly agree in times of low latency, we also expect a lot of redundant attestations. We can aggregate such a set of attestations \(I\) into a single, aggregate, attestation.
For each slot \(s\), a committee of validator \(C(s)\) is determined who is expected to attest for \(s\). Assume that two aggregate attestations were formed from validators attesting for \(s\), one aggregate of validators in set \(I \subseteq C(s)\) and the other with validators in set \(J \subseteq C(s)\). We have two cases:
- When the intersection of \(I\) and \(J\) is non-empty, we cannot aggregate the two aggregates further.
- When the intersection of \(I\) and \(J\) is empty, the two aggregates can themselves be aggregated, into one containing attestations from validator set \(I \cup J\).
In the following, we look at redundant, clashing and individual attestations.
How many individual attestations are contained in aggregates?
A fairly high number of aggregate attestations included in a block are actually individual attestations. Nonetheless, a significant number of aggregates tally up between 50 and 100 attestations.
We can plot the same, weighing by the size of the validator set in the aggregate, to count how many individual attestations each size of aggregates included.
Overall, we can plot the Lorenz curve of aggregate attestations. This allows us to find out the share of attestations held by the 20% largest aggregates.
The answer is 61%.
How much savings did aggregates provide?
We compare how many individual attestations exist to how many aggregates were included.
How many distinct individual attestations are there?
We have 11.07 times more individual attestations than aggregates, meaning that if we were not aggregating, we would have 11.07 as much data on-chain.
In how many aggregate attestations is a single attestation included?
We look at all individual attestations in our dataset, i.e., individual, unaggregated votes, and measure how many times they were included in an aggregate.
How many redundant aggregate attestations are there?
We call redundant identical aggregate attestations (same four attributes and same set of validator indices) which are included in more than one block. It can happen when a block producer does not see that an aggregate was previously included (e.g., because of latency), or simply when the block producer doesn’t pay attention and greedily adds as many aggregates as they know about.
The mode is 1, which is also the optimal case. A redundant aggregate does not have much purpose apart from bloating the chain.
How many times did a block include the exact same aggregate attestation more than once?
We could call these strongly redundant, as this is pure waste.
We see that 57 times, identical aggregates were included twice in a block.
How many times were clashing attestations included in blocks?
We define clashing attestations as two aggregate attestations included in the same block, with identical attributes (same attesting slot, beacon chain head, source block and target block). We can further define the following two notions, assuming the two aggregate attestations include attestations of validator sets \(I\) and \(J\) respectively:
- Weakly clashing: the two aggregates have different validator indices, \(I \neq J\).
- Strongly clashing: \(I \neq J\) and \(I \cap J \neq \emptyset\). The two aggregate attestations were incompatible, so could not be aggregated further.
We obtain how many times an attestation weakly clashes with itself, i.e., is included multiple times in a single block with different validator sets. For instance, if an aggregate attestation is included in the same block three times with a different set of validator indices each time, we record that this aggregate is weakly clashing three times with itself. We give below the histogram of this measure.
From the plot above, we observe that some aggregates were included over 40 times in the same block, all with different sets of validator indices. Still, most aggregates were included once or a few times.
Finding weakly clashing attestations that are not strongly clashing (i.e., which could have been aggregated further) is left for future work as it is more computationally intensive. In particular, for a set of aggregates identical up to their validator indices, one must find which have an empty overlap.
Note that optimally aggregating a set of aggregates is NP-complete! Here is a reduction of the optimal aggregation problem to the graph colouring. Set aggregate attestations as vertices in a graph, with an edge drawn between two vertices if the validator sets of the two aggregates have a non-empty overlap. In the graph colouring, we look for the minimum number of colours necessary to assign a colour to each vertex such that two connected vertices do not have the same colour. All vertices who share the same colour have an empty overlap, and thus can be combined into an aggregate. The minimum number of colours necessary to colour the graph tells us how few aggregates were necessary to combine a given set of aggregates further.